Method to combine brightfield and fluorescent channels for cell image segmentation and morphological analysis using images obtained from imaging flow cytometer (IFC)

ABSTRACT

A classifier engine provides cell morphology identification and cell classification in computer-automated systems, methods and diagnostic tools. The classifier engine performs multispectral segmentation of thousands of cellular images acquired by a multispectral imaging flow cytometer. As a function of imaging mode, different ones of the images provide different segmentation masks for cells and subcellular parts. Using the segmentation masks, the classifier engine iteratively optimizes model fitting of different cellular parts. The resulting improved image data has increased accuracy of location of cell parts in an image and enables detection of complex cell morphologies in the image. The classifier engine provides automated ranking and selection of most discriminative shape based features for classifying cell types.

RELATED APPLICATION(S)

This application is the U.S. National Stage of International ApplicationNo. PCT/US2017/035922, filed Jun. 5, 2017, which designates the U.S.,published in English, and claims the benefit of U.S. ProvisionalApplication No. 62/348,356, filed on Jun. 10, 2016. The entire teachingsof the above application(s) are incorporated herein by reference.

BACKGROUND

Cytometry is the measurement of the characteristics of cells includingcell size, cell count, cell morphology (shape and structure), cell cyclephase, DNA content, and the existence or absence of specific proteins onthe cell surface or in the cytoplasm. Cytometry is used to characterizeand count blood cells in common blood tests. In a similar fashion,cytometry is also used in cell biology research and in medicaldiagnostics to characterize cells in a wide range of applicationsassociated with diseases such as cancer and other disorders.

Cytometric devices include image cytometers which operate by staticallyimaging a large number of cells using optical microscopy. Prior toanalysis, cells may be stained to enhance contrast or to detect specificmolecules by labeling these with fluorochromes. The cells may be viewedwithin a hemocytometer to aid manual counting. The introduction of thedigital camera has led to the automation of image cytometers includingautomated image cytometers for cell counting and automated imagecytometers for sophisticated high-content screening systems.

Another cytometric device is the flow cytometer. In a flow cytometer,cells are suspended in a stream of fluid and passed by an electronicdetection apparatus. The cells are characterized optically or by the useof an electrical impedance method called the Coulter principle. Todetect specific molecules when optically characterized, cells are inmost cases stained with the same type of fluorochromes that are used byimage cytometers. Flow cytometers generally provide less data than imagecytometers, but have a significantly higher throughput.

For example, in biotechnology, flow cytometry is a laser- orimpedance-based, biophysical technology and is employed in cellcounting, cell sorting, biomarker detection and protein engineering. Theflow cytometer allows simultaneous multiparametric analysis of thephysical and chemical characteristics of up to thousands of particles(cells) per second. Flow cytometry is routinely used in the diagnosis ofhealth disorders, especially blood cancers. However flow cytometry hasother applications in research, clinical practice and clinical trials. Acommon variation is to physically sort cell particles based on theirproperties, so as to purify cell populations of interest.

Assessment of morphology is critical for identification of differentcell types. It also plays a vital role in evaluating the health of thecell. However, accurately classifying complex morphologies such assickle cells, diatoms, and spermatozoa can be a challenge due to theheterogeneous shapes within a cell.

Current methodology involves (i) obtaining images from a microscope,(ii) then manually locating the subcellular components, and (iii)estimating the coordinates for spatial location of that cellularcomponent and subcomponents in order to approximately map them back to abrightfield image.

SUMMARY OF THE INVENTION

With the present invention, Applicant addresses the shortcomings andchallenges of the art. In particular, embodiments provide a computerimplemented cell morphology classification system or tool. Embodimentsemploy a multispectral imaging flow cytometer to acquire a variety ofimages in different imaging modes, such as brightfield, side scatter,and fluorescent images, of a subject cell. The images are simultaneouslyacquired and spatially well aligned across the imaging modes. Theacquired images are fed to a processor-based classifier engine (i.e., aclassifier) that automatically analyzes thousands of cellular images andaccurately identifies different cellular and subcellular components.Specifically, the classifier engine performs multispectral segmentationusing segmentation masks for cells and subcellular parts. The classifierengine performs the multispectral segmentation by iteratively optimizingmodel fitting of different cellular parts. By using the segmentationmasks, the classifier engine extracts advanced shape features such ascontour curvature and bending score, and in turn detects complexmorphologies such as fragmented or detached cells, stretched or pointedcell boundary, etc. Next, the classifier engine provides automatedranking and selection of most discriminative shape based features forclassifying test (sample) cells into subpopulations.

One embodiment has demonstrated the efficacy of Applicant's approach onboar semen samples by accurately classifying various sperm morphologydefects. Other embodiments may classify cancer cells, sickle cells andother cell types with complex morphologies. Yet other embodimentsprovide a diagnostic tool for identifying cell morphology and/orclassifying different cell types.

Embodiments provide a computer-automated cell classification systemcomprising: an imaging flow cytometer, a classifier engine, and anoutput unit. The imaging flow cytometer acquires a multispectralplurality of images of a sample cell. The plurality of images issimultaneously acquired across multiple imaging modes, and each of theimages of the plurality is spatially well aligned with each other.

The classifier engine is executed by a computer processor and coupled toreceive the acquired multispectral plurality of images by the image flowcytometer. The classifier engine:

(A) segments one of the images of the acquired plurality intocomponents, the one image providing morphology of the sample cell, andthe components being formed of image subcomponents representing parts ofthe sample cell;

(B) improves accuracy of location of cell parts in the one image by: (i)segmenting other images of the acquired plurality into componentscorresponding to components segmented from the one image, the otherimages being of different imaging modes than the one image and eachother, and the components segmented from the other images serving asmasks of image subcomponents of the one image, (ii) spatiallycorrelating the subcomponent masks generated by the segmented otherimages to the one image, and using the subcomponent masks as foregroundobject markers of respective cell parts for the one image, and (iii)applying a graphic cut segmentation to the one image in a manner thatgenerates improved image data of the one image having increased accuracyof location of cell parts; and

(C) reprocesses the one image resulting from (B) and having thegenerated improved image data including component masks with relativepositions, the reprocessing identifying cell morphology from the oneimage and thereby classifying cell type of the sample cell. The outputunit renders indications of the identified cell morphology and/or celltype classification from the classifier engine. The output unit may beany of: a computer display monitor, a diagnostic test indicator, orother digital rendering.

In embodiments, the cell classification system employs a brightfieldimage as the one image, and the other images are fluorescent images ofdifferent fluorescent channels. In such cell classification systems, theclassifier engine step (B) of improving accuracy of location of cellparts in the brightfield image iteratively applies (i) through (iii) toeach of the different fluorescent images.

In other embodiments of the cell classification system, the classifierengine classifies cell types including any one or more of: sperm cellshaving disorders, sickle cells, cancer cells, and other cells indicativeof disease or health disorder. In some embodiments, the classifierengine may classify defective sperm cells, and the output unit is adiagnostic indicator rendering indication of infertility. The classifierengine may classify a defective sperm cell by automatically identifyingone or more of: cytoplasmic droplets, occurrence of Distal MidpieceReflex (DMR), and shape of sperm head.

Other embodiments provide a computer-based diagnostic tool foridentifying cell morphology and classifying cell type. Such a diagnostictool comprises an input assembly and a classifier engine communicativelycoupled to the input assembly. The input assembly provides access to asource of plural multispectral images of a sample cell. The pluralimages are simultaneously acquired by an imaging flow cytometer, and theplural images are spatially well aligned amongst each other.

The classifier engine receives the plural multispectral images form theinput assembly. The classifier engine is executed by a computerprocessor and processes the received images by:

(A) segmenting one of the received images into components, the one imageproviding morphology of the sample cell, and the components being formedof image subcomponents representing parts of the sample cell;

(B) improving accuracy of location of cell parts in the one image by:(i) segmenting other ones of the received images into componentscorresponding to components segmented from the one image, the otherreceived images being of different imaging modes than the one image andeach other, and the components segmented from the other received imagesserving as masks of image subcomponents of the one image, (ii) spatiallycorrelating the subcomponent masks generated by the segmented otherreceived images to the one image, and using the subcomponent masks asforeground object markers of respective cell parts for the one image,and (iii) applying a graphic cut segmentation to the one image in amanner that generates improved image data of the one image havingincreased accuracy of location of cell parts; and

(C) reprocessing the one image resulting from (B) and having thegenerated improved image data including component masks with relativepositions.

The reprocessing by the classifier engine identifies cell morphologyfrom the one image and thereby classifies cell type of the sample cell.The classifier engine provides an output indication of the identifiedcell morphology and/or cell type classification. Such output indicationserves as the diagnostic tools' indicator of infertility, defectivesperm cells, or similar analytic outcome, and may be rendered, fornon-limiting example, on a display screen.

Taking advantage of the ability of imaging flow cytometers to acquireimages at high speed and resolution and using the analysis frameworkwith specific statistical modeling, embodiments observed above 90percent sensitivity and specificity in results. As such, embodimentsdemonstrate the ability to accurately extract salient morphologicalfeatures, and classify cell types in an objective and precise manner.

BRIEF DESCRIPTION OF THE DRAWINGS

The foregoing will be apparent from the following more particulardescription of example embodiments of the invention, as illustrated inthe accompanying drawings in which like reference characters refer tothe same parts throughout the different views. The drawings are notnecessarily to scale, emphasis instead being placed upon illustratingembodiments of the present invention.

FIG. 1 is a block diagram of an embodiment of the present invention.

FIG. 2 is a flow diagram of the classifier engine of FIG. 1.

FIGS. 3A-3D illustrate masks employed by embodiments.

FIG. 4 is a schematic view of a computer network for deployingembodiments.

FIG. 5 is a block diagram of a computer node in the FIG. 4 computernetwork environment.

FIG. 6 is a graphical illustration of the PI staining method of livecell identification and quantification.

FIGS. 7(a) and 7(b) are graphs illustrating droplet detection thresholdand sperm droplets detection, respectively, using Radial SymmetryStrength (RSS) scores.

FIG. 8 is a graph of bending scores of sperm cells.

FIGS. 9a-9c are image illustrations of sperm head shapes made detectableby embodiments.

DETAILED DESCRIPTION OF THE INVENTION

A description of example embodiments of the invention follows.

The teachings of all patents, published applications and referencescited herein are incorporated by reference in their entirety.

Illustrated in FIG. 1 is a cell classification system or computer-basedtool 100 embodying the present invention. A biological sample ofinterest, such as bodily fluids or other material (medium) carryingsubject cells is provided as input to a multispectral imaging flowcytometer 105. The imaging flow cytometer 105 combines the fluorescencesensitivity of standard flow cytometry with the spatial resolution andquantitative morphology of digital microscopy. An example imaging flowcytometer is the ImageStream® by Amnis of Applicant. Other imaging flowcytometers are suitable.

Also imaging flow cytometer 105 is compatible with a broad range of cellstaining protocols of conventional flow cytometry as well as withprotocols for imaging cells on slides. See U.S. Pat. Nos. 6,211,955;6,249,341; 7,522,758 and “Cellular Image Analysis and Imaging by FlowCytometry” by David A. Basiji, et al. in Clinical Laboratory Medicine2007 September, Volume 27, Issue 3, pages 653-670 (herein incorporatedby reference in their entirety).

Imaging flow cytometer 105 electronically tracks moving cells in thesample with a high resolution multispectral imaging system andsimultaneously acquires multiple images of each target cell in differentimaging modes. In one embodiment, the acquired images 121 of a cellinclude: a side-scatter (darkfield) image, a transmitted light(brightfield) image, and several fluorescence images of differentspectral bands. Importantly, not only are the cellular images (i.e.,images of a cell) 121 simultaneously acquired but are also spatiallywell aligned with each other across the different imaging modes. Thusthe acquired darkfield image, brightfield image and fluorescence images(collectively images 121) of a subject cell are spatially well alignedwith each other enabling mapping of corresponding image locations towithin about 1-2 pixels accuracy.

The acquired cellular images 121 are output from imaging flow cytometer105 and input to computer-implemented classifier engine or modeler 107.For non-limiting example, embodiments may employ an input assembly forimplementing streaming feed or other access to the acquired images 121.Classifier engine 107 is configured to automatically analyze thousandsof cellular images 121 in near real time of image acquisition or access,and to accurately identify different cellular and subcellular componentsof the sample cells. That is, each image 121 has components representingcells in the sample, and a given component may be formed of one or moreimage subcomponents representing parts (portions) of a cell.

Briefly, from the acquired images 121, classifier engine 107 selects animage that illustrates the morphology of a cell, such as the brightfieldimage for non-limiting example. The classifier engine 107 segments thebrightfield image into components (representative of candidate cells),and likewise segments the other images of the different spectral bandsinto corresponding cellular components. Where the acquired images 121are spatially well aligned to each other, components resulting from thesegmented images serve as accurate segmentation masks for cells andsubcellular parts of the sample. The classifier engine 107 usescorresponding components and their subcomponents of the segmented imagesof the different spectral bands to iteratively optimize model fitting ofdifferent cellular parts of the brightfield image. As a result,classifier engine 107 extracts advanced shape features such as contourcurvature and bending scope from the processed brightfield image, and isthereby able to detect complex morphologies such as fragmented ordetached cells, stretched or pointed cell boundary, etc. in the samplecells. On output, classifier engine 107 (and thus system/tool 100)provides indications of identified cell morphologies and/orclassification of cell type. A computer display monitor or other outputunit may be used to render these output indications (visually, audibly,using multimedia, etc.) to an end-user.

Turning now to FIG. 2, a flow diagram of data, logic, and/or control ofone embodiment of classifier engine 107 is shown. Classifier engine 107begins with the brightfield image or equivalent image bearing morphologyof a cell in the acquired images 121. At step 125, classifier engine 107segments the cellular image and its components by using information fromthe brightfield image only.

Next, classifier engine 107 iterates through steps 127-131 for eachfluorescent channel image, i, of the acquired images 121. For a givenfluorescent channel image, step 127 segments the fluorescent channelimage to obtain a corresponding cellular image component (withsubcomponents) mask. Step 128 spatially correlates the resultingsubcomponent mask of step 127 back to the brightfield mask (the binaryimage data of the brightfield image of step 125) as shown in FIGS. 3A-3Dwith the label “BF Image Only”. Step 129 uses the subcomponent mask (ofstep 127) as foreground object seeds and selects background pixelscomplementary to this fluorescent subcomponent mask. As an option, step131 specifies the shape model for this subcomponent. Step 130 applies agraph cut segmentation algorithm, such as GrabCut known in the art, toarrive at an improved brightfield mask with accurate location ofsubcellular components (image subcomponents representing cell parts).Other graph cut segmentation methods may be employed.

Decision juncture 135 loops back to step 127 to process the nextfluorescent channel image i. Steps 127-131 repeat for each fluorescentchannel image of the acquired images 121. After the last of thefluorescent channel images has been processed, decision juncture 135continues at step 137.

Step 137, reprocesses the brightfield image with all the newly fusedsubcomponent masks generated from the above processing of thefluorescent channel images. The reprocessing of step 137 uses amulti-label graph cuts algorithm (known in the art) and creates a finalbrightfield image and its subcomponent masks with their relativepositions. FIGS. 3A-3D are illustrative where the “Final Fused” image isoutput by step 137 and the “Nuclear Fused” image results from step 130described above.

Computer Support

FIG. 4 illustrates a computer network or similar digital processingenvironment in which the present invention may be implemented.

Client computer(s)/devices 50 and server computer(s) 60 provideprocessing, storage, and input/output devices executing applicationprograms and the like. Client computer(s)/devices 50 can also be linkedthrough communications network 70 to other computing devices, includingother client devices/processes 50 and server computer(s) 60.Communications network 70 can be part of a remote access network, aglobal network (e.g., the Internet), a worldwide collection ofcomputers, a cloud computing environment, Local area or Wide areanetworks, and gateways that currently use respective protocols (TCP/IP,Bluetooth, etc.) to communicate with one another. Other electronicdevice/computer network architectures, such as an internet of things,and the like are suitable.

FIG. 5 is a diagram of the internal structure of a computer (e.g.,client processor/device 50 or server computers 60) in the computersystem of FIG. 4. Each computer 50, 60 contains system bus 79, where abus is a set of hardware lines used for data transfer among thecomponents of a computer or processing system. Bus 79 is essentially ashared conduit that connects different elements of a computer system(e.g., processor, disk storage, memory, input/output ports, networkports, etc.) that enables the transfer of information between theelements. Attached to system bus 79 is I/O device interface 82 forconnecting various input and output devices (e.g., keyboard, mouse,source feed or access to acquired images 121, displays, monitors,printers, speakers, etc.) to the computer 50, 60. Network interface 86allows the computer to connect to various other devices attached to anetwork (e.g., network 70 of FIG. 4). Memory 90 provides volatilestorage for computer software instructions 92 and data 94 used toimplement an embodiment of the present invention (e.g., classifierengine 107 code detailed above). Disk storage 95 provides non-volatilestorage for computer software instructions 92 and data 94 used toimplement an embodiment of the present invention. Central processor unit84 is also attached to system bus 79 and provides for the execution ofcomputer instructions 92 such as the classifier engine 107 programillustrated in FIG. 2.

The flow of data and processor 84 control is provided for purposes ofillustration and not limitation. It is understood that processing may bein parallel, distributed across multiple processors, in different orderthan shown or otherwise programmed to operate in accordance with theprinciples of the present invention.

In one embodiment, the processor routines 92 and data 94 are a computerprogram product (generally referenced 92), including a computer readablemedium (e.g., a removable storage medium such as one or more DVD-ROM's,CD-ROM's, diskettes, tapes, etc.) that provides at least a portion ofthe software instructions for the invention system. Computer programproduct 92 can be installed by any suitable software installationprocedure, as is well known in the art. In another embodiment, at leasta portion of the software instructions may also be downloaded over acable, communication and/or wireless connection. In other embodiments,the invention programs are a computer program propagated signal product107 embodied on a propagated signal on a propagation medium (e.g., aradio wave, an infrared wave, a laser wave, a sound wave, or anelectrical wave propagated over a global network such as the Internet,or other network(s)). Such carrier medium or signals provide at least aportion of the software instructions for the present inventionroutines/program 92.

In alternate embodiments, the propagated signal is an analog carrierwave or digital signal carried on the propagated medium. For example,the propagated signal may be a digitized signal propagated over a globalnetwork (e.g., the Internet), a telecommunications network, or othernetwork. In one embodiment, the propagated signal is a signal that istransmitted over the propagation medium over a period of time, such asthe instructions for a software application sent in packets over anetwork over a period of milliseconds, seconds, minutes, or longer. Inanother embodiment, the computer readable medium of computer programproduct 92 is a propagation medium that the computer system 50 mayreceive and read, such as by receiving the propagation medium andidentifying a propagated signal embodied in the propagation medium, asdescribed above for computer program propagated signal product.

Generally speaking, the term “carrier medium” or transient carrierencompasses the foregoing transient signals, propagated signals,propagated medium, storage medium and the like.

Example

Assessment of sperm morphology is one of the most important steps in theevaluation of sperm quality. A higher percentage of morphologicallyabnormal sperm is strongly correlated to lower fertility. Morphologicalimage analysis can be used to segment sperm morphology, extractassociated quantitative features, and classify normal and abnormalsperms in large quantities. In the majority of human infertility clinicsand livestock semen laboratories, brightfield microscopy is used toevaluate sperm morphology. However, since sperm morphology contains avariety of shapes, sizes, positions, and orientations, the accuracy ofthe analysis can be less than optimal as it is based on a limited numberof cells (typically 100-400 cells per sample). To overcome thischallenge, we used ImageStream imaging flow cytometry to acquirebrightfield, side-scatter and Propidium Iodide (PI) images of boar spermsamples at 60× magnification. We developed novel image algorithms toperform image segmentation and detect abnormal sperm cells using salientshape descriptors (invariant of scale, position, and orientation), suchas diameter, circularity, elongation, corners, and negative curvatures.Taking advantage of the ability of the imaging flow cytometer to acquireimages at high resolution and speed, we demonstrate the validity ofusing image based parameters that can be adapted to each spectral imageand features to assess sperm morphology with statistical significance inan objective manner.

Materials & Methods

Semen Collection and Processing

Boars (8-26 months old) were used in this study. Semen samples werecollected and fixed with 1% formalin. For the live cell analysis, frozenstraws stored in liquid nitrogen were thawed at 37 degrees for 30seconds and labelled with PI for 10 minutes.

Image Acquisition

Sperm samples were run on Amnis® ImageStream^(X) MkII imaging flowcytometer. Around 5,000-10,000 brightfield, side-scatter and PropidiumIodide (PI) events were acquired at 60× magnification at approximately400 objects per second.

Image Analysis Using IDEAS®

Image Analysis was performed using novel Image-based algorithms inIDEAS® (by Amnis) image analysis software, an embodiment of Applicant'sabove described system and method involving classifier engine 107.

Results

Identification of Live Cells

PI stain was used to assess the membrane integrity of the sperm toquantify the proportion of live and dead cells in order to select sireswith best fertilizing capacity.

However, as shown in FIG. 6, PI staining alone was not sufficient toselect most suitable sires since the PI negative live cells could havemorphological abnormalities which could affect the fertility of theboar.

Once the proportion of live cells was calculated, then sperm morphologywas examined using image based morphological features to evaluate thehealth of the sperm.

Identification of Cytoplasmic Droplets

One of the most common abnormalities in boar semen is the presence ofcytoplasmic droplets. Their presence can be a sign of immature sperm orindicate a lack of synchrony between sperm production and semencollection from the boar. In humans, if the proximal droplets are themajor site of water influx in hypo-osmotic solutions and if volumeregulation during regulatory volume decrease (RVD) can't occur, theirsize may affect the penetration into female tract. So identification ofthe presence of droplets could be of diagnostic importance forinfertility.

These droplets (both distal and proximal) contribute to approximately 15percent of the total morphological abnormalities in boar sperm. Toextract these cytoplasmic droplets using brightfield morphology, an RSS(Radial Symmetry Strength) score feature was created in IDEAS software(generally an embodiment of 107). The prevalence of droplet correlateddirectly with the droplet spatial feature extracted from the imaging.FIGS. 7(a) and 7(b) are illustrative where FIG. 7(a) shows a model tocalculate RSS threshold for droplet detection. The gamma distribution ofthe droplets is fitted to the RSS scores and a detection threshold (theRSS Threshold) results. FIG. 7(b) graphs sperm by RSS score. For thesesperm with RSS scores above the 1.2024 threshold (calculated in FIG.7(a)), various cytoplasmic droplets were detected and droplet spatialfeatures extracted from the subject images. In a sample of 2000 sperms,339 sperms were detected with cytoplasmic droplets which is a 16.95%detection percentage.

Furthermore, with the spatial position of sperm head and midpiece beinglocated in the above multispectral fusion process, the subpopulation ofproximal droplets that is at the vicinity of sperm midpiece and distaldroplets which is close to sperm flagellum can be statisticallydetermined, which is a very important differentiation in terms ofidentifying potential sperm cell abnormality and link to infertility.Thus in embodiments, the resulting improved image data has increasedaccuracy of location of cell parts in an image and enables detection ofcomplex cell morphologies in the image.

Identification of Bent Tail and Distal Midpiece Reflex

Abnormalities in the midpiece and tail generally arise with defects inthe spermatogenesis. Occurrence of Distal Midpiece Reflex (DMR) or benttail may result in non-motile or abnormal motility in sperm.Consequently the presence of such abnormalities is generally associatedwith subfertility.

Sperms with bent or curved tails can account for about 10 percent of thetotal morphological abnormalities in boars. To accurately quantify thenumber of sperm with DMR or curved tails, a new “bending feature” wasdeveloped in IDEAS software embodying classifier engine 107. The bendingfeature is constructed to directly characterize the bending severity bythe curvature maximum (clockwise or counter-clockwise) along the spermskeleton mask. FIG. 8 is a graph of the bending scores of cells. Thecell numbers and thresholding line T in FIG. 8 are computed by the meanof the mean, median, and first quartile of the bending scores.

Identification of Sperm Head

Abnormalities in sperm head morphology could impair both fertilizationrate and the subsequent embryonic development with failure of cleavagebeing the primary outcome.

Sperms with abnormal heads account for about 4-5 percent of the totalmorphological abnormalities in boars. To accurately identify the spermhead, a new shape-based thresholding mask was developed in IDEASsoftware (embodying classifier 107). FIGS. 9(a)-9(c) show sperm headshapes, the detection of which is enabled by embodiments of the presentinvention. FIG. 9(a) is an image of a pyriform head shape. FIG. 9(b) isan image of an elongated head shape. FIG. 9(c) is an image of amisshapen head shape. Embodiments produce improved image data havingincreased accuracy of location of call parts in the image. Using the newmask, contour features were calculated along the head to measure theabnormalities in sperm head.

Statistics

Table 1 summarizes the foregoing morphological features of boar semenmade detectable in improved image data by embodiments of Applicant'sclassifier engine 107 detailed above. In particular, the improved imagedata of Applicant's system 100 has increased accuracy of location ofcell parts in an image and enables detection of complex cellmorphologies in the image, such as shown in Table 1.

TABLE 1 Sensitivity and Specificity results for the morphologicalfeatures Feature Sensitivity Specificity Identification of cytoplasmic90 100 droplet Identification of bent tail 85 90 Identification of spermhead 99 98

CONCLUSION

Studies have shown that there is a direct correlation between themorphology of the sperm and the health of the sperm. Currently acomplete analysis of morphology is done by microscopy which can be timeconsuming and requires a trained staff. Using Amnis® ImageStream^(X)MkII imaging flow cytometer's ability to acquire images at highresolution and speed and the morphological parameters in IDEAS software(classifier engine 107), we have successfully demonstrated highspecificity and sensitivity in identifying the most prevalentabnormalities in boar semen with minimal use of fluorescent staining.

While this invention has been particularly shown and described withreferences to example embodiments thereof, it will be understood bythose skilled in the art that various changes in form and details may bemade therein without departing from the scope of the inventionencompassed by the appended claims.

For example, the foregoing discussion uses the brightfield image as theprimary working image data improved upon by the Applicant's iterativeprocess of FIG. 2. Other images bearing or representing morphology ofcells are suitable instead of the brightfield image.

In another example, in addition to or alternative to sperm cells, othercells such as cancer cells, cells indicative of disease or disorder,sickle cells, and the like may be the subject of embodiments of thepresent invention. In like manner, diagnostic tools for identifying cellmorphology or classifying cell type may embody the foregoing describedprinciples of the present invention.

The embodiment of FIG. 2 employs iterative processing of themultispectral images. Other image processing approaches (parallelprocessing, distributed processing, etc.) may be employed minding theprinciples of the present invention to provide relatively fast (nearreal time) and accurate results (identification of cell morphologyand/or classification of cell types).

What is claimed is:
 1. A computer-automated cell classification systemcomprising: an imaging flow cytometer acquiring a multispectralplurality of images of a sample cell, the plurality of images beingsimultaneously acquired across multiple imaging modes, and each of theimages of the plurality being spatially well aligned with each other; acomputer processor, which executes a classifier engine coupled toreceive the acquired multispectral plurality of images, the classifierengine: (A) segmenting one of the images of the acquired plurality intocomponents, the one image providing morphology of the sample cell, andthe components being formed of image subcomponents representing parts ofthe sample cell; (B) improving accuracy of location of cell parts in theone image by: (i) segmenting other images of the acquired plurality intocomponents corresponding to components segmented from the one image, theother images being of different imaging modes than the one image andeach other, and the components segmented from the other images servingas masks of image subcomponents of the one image, (ii) spatiallycorrelating the subcomponent masks generated by the segmented otherimages to the one image, and using the subcomponent masks as foregroundobject markers of respective cell parts for the one image, and (iii)applying a graph cut segmentation to the one image in a manner thatgenerates improved image data of the one image, wherein the improvedimage data is improved by having increased accuracy of location of cellparts; and (C) reprocessing the one image resulting from (B) and havingthe generated improved image data including component masks withrelative positions, the reprocessing identifying cell morphology fromthe one image and thereby classifying cell type of the sample cell; anda display screen rendering indications of the identified cell morphologyand/or cell type classification from the classifier engine.
 2. The cellclassification system of claim 1 wherein the one image is a brightfieldimage and the other images are fluorescent images of differentfluorescent channels.
 3. The cell classification system of claim 2wherein the step (B) of improving accuracy of location of cell parts inthe brightfield image iteratively applies (i) through (iii) to each ofthe different fluorescent images.
 4. The cell classification system ofclaim 1 wherein the classifier engine classifies cell types includingany one or more of: sperm cells having disorders, sickle cells, cancercells, and other cells indicative of disease or health disorder.
 5. Thecell classification system of claim 1 wherein the display screen is acomputer display monitor.
 6. The cell classification system of claim 1wherein the classifier engine classifies defective sperm cells, and thedisplay screen renders an indication of infertility.
 7. The cellclassifier of claim 6 wherein the classification engine classifies adefective sperm cell by automatically identifying one or more of:presence of cytoplasmic droplets, occurrence of Distal Midpiece Reflex(DMR), and shape of sperm head.
 8. A computer-implemented method ofidentifying cell morphology and classifying cell types, comprising:using an imaging flow cytometer, acquiring a multispectral plurality ofimages of a sample cell, the plurality of images being simultaneouslyacquired across multiple imaging modes, and each of the images of theplurality being spatially well aligned with each other; in a digitalprocessor, responsively processing the acquired multispectral pluralityof images by: (A) segmenting one of the images of the acquired pluralityinto components, the one image providing morphology of the sample cell,and the components being formed of image subcomponents representingparts of the sample cell; (B) improving accuracy of location of cellparts in the one image by: (i) segmenting other images of the acquiredplurality into components corresponding to components segmented from theone image, the other images being of different imaging modes than theone image and each other, and the components segmented from the otherimages serving as masks of image subcomponents of the one image, (ii)spatially correlating the subcomponent masks generated by the segmentedother images to the one image, and using the subcomponent masks asforeground object markers of respective cell parts for the one image,and (iii) applying a graph cut segmentation to the one image in a mannerthat generates improved image data of the one image, wherein theimproved image data is improved by having increased accuracy of locationof cell parts; and (C) reprocessing the one image resulting from (B) andhaving the generated improved image data including component masks withrelative positions, the reprocessing identifying cell morphology fromthe one image and thereby classifying cell type of the sample cell; andoutputting an indication of the identified cell morphology and/or celltype classification.
 9. The method of claim 8 wherein the one image is abrightfield image and the other images are fluorescent images ofdifferent fluorescent channels.
 10. The method of claim 9 wherein thestep (B) of improving accuracy of location of cell parts in thebrightfield image iteratively applies (i) through (iii) to each of thedifferent fluorescent images.
 11. The method of claim 8 wherein thereprocessing classifies cell types including one or more of: sperm cellshaving disorders, sickle cells, cancer cells, and other cells indicativeof disease or health disorder.
 12. The method of claim 8 wherein theoutput indication is a diagnostic test result indicator.
 13. The methodof claim 8 wherein the responsively processing includes classifyingdefective sperm cells; and the outputting renders an indication ofinfertility.
 14. The method of claim 13 wherein the classifying adefective sperm cell includes the digital processor automatingidentification of one or more of: presence of cytoplasmic droplets,occurrence of Distal Midpiece Reflex (DMR), and shape of sperm head.